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Abstract 

Background Many clinicians depend solely on journal 
abstracts to guide clinical decisions. 
Objectives This study aims to determine if there are dif- 
ferences in the accuracy of responses to simulated cases 
between resident physicians provided with an abstract 
only and those with full-text articles. It also attempts to 
describe their information-seeking behaviour. 
Methods Seventy-seven resident physicians from four 
specialty departments of a tertiary care hospital com- 
pleted a paper-based questionnaire with clinical simula- 
tion cases, then randomly assigned to two intervention 
groups— access to abstracts-only and access to both 
abstracts and full-text. While having access to medical 
literature, they completed an online version of the same 
questionnaire. 

Findings The average improvement across departments 
was not significantly different between the abstracts- 
only group and the full-text group (p=0.44), but when 
accounting for an interaction between intervention and 
department, the effect was significant (p=0.049) with 
improvement greater with full-text in the surgery 
department. Overall, the accuracy of responses was 
greater after the provision of either abstracts-only or 
full-text (p<0.0001). Although some residents indicated 
that 'accumulated knowledge' was sufficient to respond 
to the patient management questions, in most instances 
(83°/o of cases) they still sought medical literature. 
Conclusions Our findings support studies that doctors 
will use evidence when convenient and current evidence 
improved clinical decisions. The accuracy of decisions 
improved after the provision of evidence. Clinical deci- 
sions guided by full-text articles were more accurate 
than those guided by abstracts alone, but the results 
seem to be driven by a significant difference in one 
department. 

Introduction 
Background 

Broad estimates show that a specialist would need 
almost two million pieces of data to practice good medi- 
cine. 1 To keep updated and apply evidence-based medi- 
cine (EBM) in practice, a physician must critically 
appraise full-text articles to guide their clinical decision 
making. 2 However, owing to limited access to full-text 
articles, inadequate critical appraisal skills or lack of 
time to read the entire article, many clinicians depend 
solely on journal abstracts to answer clinical 



questions. 3-8 Journal abstracts may have become the de 
facto resource for health professionals wanting to prac- 
tice EBM because they are easy to read and are easily 
accessible anywhere. 2 5 7 9 

Although abstracts are commonly utilised for clinical 
decisions, caution should be made in using them 
because they may not completely reflect the entire 
article. 7 Studies by Pitkin ef al 10 11 and Peacock ef a/ 12 
identified abstracts that contained data which were dif- 
ferent or missing in the full-text. High-impact factor 
journals had abstracts that failed to include harm 
despite being mentioned in the main article. 13 A study 
by Berwanger ef at found that the abstracts of rando- 
mised controlled trials from major journals were 
reported with suboptimal quality. Moreover, abstracts 
are also subject to authors' biases which may mislead 
the readers. 14 

Efforts have been made to improve the quality and 
accuracy of journal abstracts since they are often the 
most commonly read part of an article— if not the only 
part read. 10 11 15 In 1987, the Ad Hoc Working Group 
for Critical Appraisal of the Medical Literature intro- 
duced a seven-heading format (Objectives, Design, 
Setting, Patients, Interventions, Measurements and 
Conclusion) for structured abstracts. 16 Variations in 
structured abstracts include the eight-heading format 
proposed by Haynes ef al," IMRAD 18-20 (Introduction, 
Methods, Results and Discussion), and more recently, 
BMfs pico format 21 (Patient, Intervention, Comparison 
and Outcome). Structured abstracts tend to be longer 
than traditional ones but they also tend to have better 
content, readability, recall and retrieval. 14 18 22-25 Aside 
from structuring, 'quality criteria' and guidelines have 
been developed to assist authors in preparing 
abstracts. 26 27 

Most of the research on journal abstracts focus on 
their quality compared to the full-text, 10 11 13 22 23 or 
based on their structure. 14 20 24 Given the tendency of 
physicians to use abstracts for evidence, there is a need 
to evaluate their reliability in clinical decision making. 
A study by Barry ef al 5 looked at the effect of abstract 
format on physicians' management decisions. However, 
we are unable to find studies that compare clinical deci- 
sions between those with access to abstract-only or 
full-text. 

The primary objective of this study was to determine 
whether there is a significant difference in the accuracy 
of the clinical decisions made on simulated cases by 
residents with access to full-text articles and those with 
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access to abstract-only. The specific objectives were: 
(1) to compare the effect of access to abstracts-only or 
full-text articles on the clinical decision-making of resi- 
dents; (2) to determine whether providing either the 
abstract or full-text article increased the accuracy of clin- 
ical decisions and (3) to characterise the information- 
seeking behaviour and use of information resources by 
residents of four departments in a tertiary care hospital. 

Methods 
Ethics review 

The research protocol was submitted for technical review 
to the Research Grants Administration Office of the 
University of the Philippines Manila and for ethical 
evaluation to the Institutional Review Board, both of 
which approved the study. 

Prestudy clinical case development 

A physician consultant from each of four clinical 
departments (Surgery, Internal Medicine, Emergency 
Medicine and Family and Community Medicine) pre- 
pared five simulated clinical cases of varying complexity 
and the corresponding clinical questions to assess the 
residents' management decisions. They searched 
PubMed for at least three recent (from 2007 onwards) 
journal articles that were deemed relevant for each case. 
'Gold standard' answers to the clinical questions were 
based on the journal articles and other relevant infor- 
mation (applicability and appropriateness to local condi- 
tions, available resources and practice environments). A 
paper-based questionnaire was used for the preinterven- 
tion assessment while an online version was used 
during the intervention phase to allow access to journal 
abstracts or full-text articles. 

Study participants and setting 

Seventy-seven resident physicians from the four clinical 
departments (above) at the Philippine General Hospital 
participated in the study. The Philippine General 
Hospital is a 1500-bed tertiary care, state-owned, refer- 
ral center and teaching hospital of the University of the 
Philippines College of Medicine, College of Nursing, 
College of Dentistry and allied colleges. It is the largest 
government hospital with a yearly patient load of 
600 000, mostly indigent patients. Fourteen clinical 
departments offer residency and fellowship training. 

Study design 

During the prestudy briefing, the residents were informed 
that they were to answer questions related to the case 
simulations and that they could access reference articles 
if needed during the online phase of the study. Written 
consent was obtained and paper-based case simulations 
were given to each resident to replicate the hospital scen- 
ario of paper patient records. After reading the case simu- 
lations, they were asked to respond to five clinical 
questions and indicate whether they considered a litera- 
ture search was needed to answer the questions or 
accumulated knowledge 28 was adequate. Accumulated 
knowledge was defined in this study as the residents' per- 
sonal knowledge base accumulated through years of 
formal education, training, research of the medical litera- 
ture and clinical experience. Immediately after the 



preintervention phase, the residents were randomly 
assigned to one of two groups-access to 'full-text' or 
'abstracts-only,' stratified by department. The same clin- 
ical cases and questions in the preintervention phase 
were presented to the residents using the online version 
of the questionnaire to simulate real-time access to 
medical literature. A 20-min time limit was allotted for 
each question both for the paper-based and online ques- 
tionnaire. The journal material provided, whether 
abstracts-only or full-text, was dependent on their 
assigned group. If the resident assigned to the 
abstracts-only group clicked on the link to the full-text, a 
prompt saying, 'Sorry, full-text is unavailable' appeared. 
Although access to either journal abstracts or full-text 
articles on the online version was available to all resi- 
dents, they had the option of not using any resource at 
all. The residents' actions regarding the use or non-use of 
medical literature were recorded. Mouse clicks related to 
the residents' request for the articles' abstracts or full-text 
were logged in the server. The accuracy of response was a 
measure of correctness of residents' answer when com- 
pared with the answers ('gold standard') provided by the 
consultants. The same consultants who prepared the clin- 
ical cases and questions evaluated the accuracy of the 
residents' answers. A correct response was scored as '1' 
and an incorrect response scored '0'. Incomplete 
responses were rated as inaccurate and scored '0'. 
Resident responses were anonymised in both the paper 
and online versions. 

Data analysis 

In order to account for the repeated measures nature of 
the data (physicians answered multiple questions), we 
fit mixed effects logistic regression models with depart- 
ment type, intervention, and the interaction between 
department and intervention as independent variables, 
and accuracy of the response as the dependent variable. 
Unless otherwise stated, all results were based on this 
model. Resident year level was also considered as a 
predictor in the model but was not found to be signifi- 
cant and was dropped. We also fit a model with an 
interaction between intervention and department. For 
univariate analysis, we used the nonparametric 
Wilcoxon two-sample tests and Fisher exact tests, as 
appropriate. 29 All analyses were performed using R 
Statistical Software. 30 

Results 

Participant profile 

Seventy-seven residents from the departments of 
Surgery (n=20), Internal Medicine (n=20), Emergency 
Medicine (n=20) and Family and Community Medicine 
(n=17) participated in this study. Table 1 shows the 
description of the study participants by department. 

Comparing the effect of abstract-only and full-text 
access on the accuracy of clinical decision making 

The first objective was to answer the question: Is there a 
significant difference in the accuracy of responses of 
residents in the abstract-only group and full-text group? 
Overall, there was no significant difference between the 
interventions (p = 0.44). Post-hoc power of the experi- 
ment to detect an overall difference between the 
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; resident trainees by department 




Surgery, 


Internal medicine, 


Emergency medicine, 


Family and community medicine, 


("harartpri^tir^ n=9fl 

\_lld 1 uLlCI 1 JUL J 1 1 — Z. \J 


n=20 


n=20 


n=17 


Mean age, years (SD*) 29.2 (2.7) 


28.1 (1.6) 


30.8 (3.3) 


30.5 (2.4) 


Gender, n (%) 








Female 5 (25) 


3 (15) 


6 [30 J 


13 (76) 


Male 15 (75) 




14 (70) 


4 (24) 


Years in residency training, n (%) 








1 6 (30) 


7(35) 


7(35) 


11 (65) 


2 2 (10) 


6(30) 


7(35) 


1 (6) 


>3 12 (60) 


7(35) 


6(30) 


5(29) 



*Standard deviation. 



interventions was low and varied from approximately 
44-58°/o, 31 depending on the level of correlation 
between the answers within each department. In a 
model fit to include an interaction between intervention 
and department, the interaction was significant suggest- 
ing that intervention effects differed by department 
(p = 0.03). In that model, access to full-text was signifi- 
cantly better than access to abstracts-only (p=0.049). 

We then compared the effect of the interventions 
within departments in order to investigate which depart- 
ments seemed to respond differently to the others with 
respect to the effect of the interventions. We found no 
significant difference between the interventions for the 
Internal Medicine, Emergency Medicine and Family 
Medicine departments (p=0.73, 0.13 and 0.37, respect- 
ively), but there was a difference between the interven- 
tions for the Surgery department (p=0.02). The OR for 
each department is given in table 2. The full-text group 
had 3.6 times the probability of getting a correct answer 
on a case simulation compared to the abstract-only 
group. Note that the CI for Surgery does not include 1.0, 
which indicates a significant difference. There were no 
differences found between the interventions for any 
other department. Power to detect a difference between 
the interventions in a specific department was low 
(approx. 17%) because of the reduced sample size in 
each group (n=10). We also investigated whether resi- 
dent year was a significant predictor of clinical decision 
accuracy, but it was not significant in any model. 

Accuracy of clinical decisions before and after access 
to literature search 

We calculated the mean percentage of accurate 
responses to the simulated clinical questions before and 
after each intervention. Overall, mean accuracy 
increased from 42°/o to 68% for the abstract-only inter- 
vention, and 48-75% for the full-text intervention. The 
differences between the scores before and after the two 



Table 2 Estimates of the odds of getting 


; an accurate 


response after full-text intervention compared to 


abstract-only intervention 




Department 


OR (95% CI) 


Surgery 


3.6 (1.3 to 10.3) 


Internal medicine 


0.8 (0.3 to 2.2) 


Emergency medicine 


0.5 (0.2 to 1.3) 


Family medicine 


1.6 (0.6 to 4.7) 



interventions were significant (p<0.0001). Table 3 shows 
the comparison of these percentages by department and 
the tests of significance. 

When given full-text articles, the departments of 
Surgery, Internal Medicine and Family Medicine showed 
significant improvements (p=0.003, 0.03 and <0.0001, 
respectively), while there was no change for the depart- 
ment of Emergency Medicine (p = 1.0). The differences 
among the departments were significant (p < 0.0001) for 
full-text intervention group. This suggests that full-text 
was more effective for Surgery, Internal Medicine and 
Family Medicine, but not in the Emergency Medicine 
department. However, the sample size was small (n=10 
or less) at this level. The effect of the abstract-only 
intervention seems to have been in a similar direction 
for all the departments, and no significant difference in 
effects across departments was detected. 

Information-seeking trends of the residents 

The majority of the residents (86%) indicated that the arti- 
cles provided in the online version were adequate to 
answer their questions and 77% indicated that they had 
actually read the articles. When asked whether they used 
abstracts-only or full-text articles to answer clinical ques- 
tions in actual practice, 53 of the 77 residents (69%) indi- 
cated that they relied on abstracts most of the time, while 
only 24 (3 1%) said they would read the full-text article. 

Residents were asked whether or not they felt they 
needed extra information in order to answer the ques- 
tion correctly. We recorded whether they clicked on the 
links for the abstract or the full-text. We wanted to 
answer the question: does a perceived need for more 
information correlate with how often the physicians 
actually accessed the links for abstract-only or full-text? 
For the 157 cases where the resident indicated that they 
did not require additional information, there were 131 
(83%) instances where literature was actually accessed 
(95% CI 77% to 88%). In contrast, out of the 228 cases 
where residents indicated that they needed additional 
information, there were only 12 (5%) cases where they 
did not actually access literature (95% CI 3% to 9%). 
Table 4 shows a summary of whether the resident 
requested additional information and whether they actu- 
ally accessed literature. 

Discussion 

The main question we wanted to address in this study 
was whether there is a significant difference in the 
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Table 3 Comparison of the average percentage of accurate of responses before and after interventions and tests of 
significance 







Average % 


Average % 




Test of 


Does the 






of accurate 


of accurate 




significance 


amount of 






responses 


responses 


Difference in 


for each 


improvement 






before 


after 


% be to re 


department 


differ by 






intervention 


intervention 


and after 


and overall 


department? 




N 


(SD) 


(SD) 


(95% CI)* 


(p value)t 


(p value)t 


Abstract-only (intervention 1) 














Department 














Surgery 


10 


48 (27) 


62 (15) 


14 (0 to 40) 


0.16 




Internal medicine 


10 


36 (18) 


66 (16) 


30 (20 to 60) 


0.003 


0.06 


Emergency medicine 


10 


54 (21) 


72 (14) 


18 (0 to 40) 


0.06 




Family medicine 


8 


25 (14) 


75 (14) 


50 (40 to 60) 


<0.0001 




Overall 


38 


42 (23) 


68 (15) 


27 (20 to 40) 


<0.0001 




Full-text (intervention 2) 














Department 














Surgery 


10 


60 (13) 


86 (0.16) 


26 (20 to 40) 


0.003 




Internal medicine 


10 


46 (16) 


68 (0.1) 


22 (0 to 40) 


0.03 


0.0001 


Emergency medicine 


10 


64 (25) 


64 (0.23) 


0 (-20 to 20) 


1.0 




Family medicine 


9 


20 (14) 


82 (0.21) 


62 (40 to 80) 


<0.0001 




Overall 


39 


48 (24) 


75 (0.2) 


27 (20 to 40) 


<0.0001 





*95% CI based on Wilcoxon test. 

tBased on values from the logistic regression mixed model. 



clinical decisions between residents who have access to 
abstracts-only and those with access to full-text articles. 
Overall, our results seem to demonstrate no difference in 
the accuracy of responses between residents provided 
with full-text articles and those with abstracts-only 
(p=0.4415). When we consider the clustering of physi- 
cians by department, we found a difference between the 
two interventions (p=0.0494) but further analysis 
showed that this difference was observed only in the 
department of Surgery (p=0.016). The effects of 
abstracts-only and full-text were not significantly dif- 
ferent for the Internal Medicine, Emergency Medicine 
and Family Medicine departments. However, the study 
had low power to detect differences between the inter- 
ventions within a department. 

Our study provides preliminary but useful informa- 
tion related to the use of journal abstracts in evidence- 
based practice. We believe this to be the first report 
involving physicians, that attempted to evaluate how 
abstracts measure up to full-text articles in guiding clin- 
ical decisions. This finding offers support for using 'con- 
sensus abstracts' (concurring and corroborating abstracts 
from independently conducted randomised clinical 
studies and systematic research from meta-analysis and 
systematic reviews that form the basis of clinical evi- 
dence) as a possible alternative when access to full-text 
is limited or in other circumstances when it is not 



Table 4 Residents' perceived need for additional 
information and actual access of literature 





Was literature actually 




accessed? 




No Yes 


Perceived need for more 


No 26 (17%) 131 (83%) 


information? 


Yes 12 (5%) 216 (95%) 



Row percentages are displayed. 



feasible. 2 However, clinicians who want to practice EBM 
will also find online many summaries, reviews and pre- 
appraised free resources (TRIP Database, ACP Journal 
Club, Cochrane Library, etc) or by subscription 
(UpToDate, 5-Minute Clinical Consult, etc). EBM web- 
sites will have links to these. Many of these resources 
will have applications for mobile devices like the iPhone 
or Android devices. Our observations set the stage for 
further research on the role of using abstracts in 
evidence-based practice. Future studies may include ran- 
domised controlled trials with real-time clinical 
decision-making encountered at the bedside. 

EBM encourages the use of timely and relevant 
information to complement the clinical acumen of clini- 
cians. 32 We found that the average improvement in the 
accuracy of responses across all the departments when 
either abstracts or full-text articles were provided was 
significant (p<0.0001 for both interventions). This 
finding supports previous research regarding the role of 
medical literature in improving clinical decisions. 33-35 
However, when individual departments were considered, 
there seems to be a significant difference between the 
departments in the full-text intervention group 
(p=0.0001). This difference in the effect of full-text 
between the departments appears to be due to the fact 
that there was no change in the accuracy of responses of 
Emergency Medicine residents compared to the increase 
in scores for the other residents when full-text was pro- 
vided. This may mean that full-text articles were benefi- 
cial to Surgery, Family Medicine and Internal Medicine 
residents but did not benefit Emergency Medicine resi- 
dents. A possible explanation for this is that the 
Emergency Medicine department is fast paced and resi- 
dents may not have the time to read the full-text article. 
This hypothesis was further supported by data for the 
abstract-only group where we found no significant dif- 
ference between the departments on how the 
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intervention improved the accuracy of the responses hy 
residents. 

Our study also demonstrated some trends in 
information-seeking and utilisation of evidence by resi- 
dents when presented with clinical questions. We 
observed that although residents indicated that accumu- 
lated knowledge was sufficient to answer the questions, 
in most instances (83.4%), they still accessed the 
medical literature provided. This observation supports 
earlier studies that health professionals will use evidence 
from the literature when they are easily accessible at the 
time the question arises. 37 

More than a third of the residents (68.8%) who parti- 
cipated in this study claimed that they commonly used 
abstracts in seeking answers to their clinical dilemma. 
Other studies have reported similar observations. A 
study by Haynes er aP found that two-thirds of clinical 
decisions were influenced by literature even if the full- 
text was not read. Moreover, internists reported that in 
63% of the articles they come across, only the abstracts 
were read. 4 These findings may even be higher among 
physicians in low- and middle-income countries 
because of even more limited availability of full-text 
articles. 

Limitations 

The small sample of residents from a tertiary govern- 
ment hospital in the Philippines limits the generalisabil- 
ity of the study to the larger medical community. 
Simulated clinical cases were used as surrogate to actual 
clinical encounters that a resident may be presented 
with. The clinical questions were specific within the 
realm of the disciplines and are not necessarily compar- 
able to each other. The residents only answered five 
questions which reduced the variation in the study. A 
'learning effect' was considered to explain the higher 
score during the intervention phase but was deemed 
unlikely because of the short interval period-the resi- 
dents took the online version questions immediately 
after the preintervention session. Furthermore, this study 
does not address whether access to full-text would have 
more impact than access to the abstract in a complex 
case, a case in which the details of a treatment or 
outcome or magnitude or significance might affect prac- 
tice. It also does not address the impact on standard or 
routine or long-term practice. Finally, although there 
was reasonable power to detect a difference between the 
interventions overall, there was low power to detect dif- 
ferences within a department. It is possible that there 
were differences between the interventions for each 
department but our study did not have enough sample 
size to investigate the effect of the intervention at the 
department level. 

Conclusion 

In this study, we demonstrated that clinical decisions 
made by residents improved when evidence, either 
abstracts or full-text articles were provided. However, 
this study also indicates that some clinical questions 
may be simple enough; answered quickly using accumu- 
lated knowledge, but accumulated knowledge was 
enhanced by the use of appropriate medical information. 
The residents, in spite of initially stating that 



accumulated knowledge was adequate to answer clinical 
questions, accessed evidence anyway. This confirms pre- 
vious findings that easy availability of evidence 
encourages the practice of evidence-based medicine. 
When clustered by department, clinical decisions guided 
by full-text articles were more accurate than those 
guided by abstracts alone, but this difference can be 
largely attributed to a significant difference in Surgery. 
It may be less or not at all in the other three depart- 
ments but the analysis is not conclusive because of the 
limited power of this study. Without departmental clus- 
tering, the findings seem to show that they may not be 
significantly different. 
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